The present invention relates to a movement detecting apparatus for detecting an amount of movement of an image of a subject on an image forming surface by taking in images continuously by using an imaging apparatus such as a video camera. The detected amount of movement is used for correcting blur of the video camera caused by unintentional movement of the hands or for combining the taken-in images.
Hitherto, a representative point method is used as means for observing an amount of movement from images taken in continuously. This method is used primarily to compensate movements such as blur of a video camera caused by unintentional movement of the hands. FIG. 10 is a conceptual drawing of the representative point method. It is a method in which representative points are set at fixed positions of an image of a previous frame with respect to images taken in continuously, a correlation operation and an accumulative addition operation are implemented for corresponding pixels while shifting in, two-dimensionally with respect to an image of a current frame and an amount of a shift in which values of the operations are highest is detected as an amount of movement.
It has been arranged so as to improve its accuracy by applying a Laplacian filter or the like on the set representative points to find gradient of luminance and to calculate by using that value as disclosed in Japanese Unexamined Patent Publication JP-A 6-86149.
It is also possible to detect an amount of movement by employing a block matching method of deciding positions to be combined by implementing correlation operation on parts where images overlap. FIG. 11 is a conceptual drawing of the block matching method. This is a method of setting a specific area to be referenced of an image of a previous frame and of implementing correlation operation while shifting in two-dimensional directions with respect to an image of a current frame to find an amount of movement in the same manner with the representative point method. However, while the representative point method requires to find the correlation of just several points to implement the cumulative addition, the block matching method requires to implement cumulative addition of the whole specific area, thus requiring higher speed processing.
However, in the above-mentioned representative point method, there must be certain luminance gradient always with respect to a certain representative point. For instance, when a subject is monochromatic data such as document, there has been a problem that an amount of movement cannot be detected properly because the luminance gradient is low at all representative points.
That is, although it is effective when the luminance gradient is uniformly distributed in an object image, it is difficult to detect an amount of movement in a document or the like abundant in a background for example because the part where the luminance gradient is low becomes the background.
Meanwhile, the block matching method has had a problem that it is difficult to detect an amount of movement on real-time because it takes large amounts of operations.
Accordingly, it is an object of the invention to provide a movement detecting apparatus which is capable of finding an amount of movement with good luminance and at high speed in an object image, e.g., a document image in which the luminance gradient is not uniform.
According to certain example embodiments of this invention, a movement detecting apparatus which detects an amount of movement by storing images inputted in time sequence, comprises feature point extracting means for extracting feature points of spots having a large change in brightness from an image of a current frame, search area deciding means for deciding a predetermined area of an image of a subsequent frame as a search area for correlation operation and correlation operation means for taking a correlation between the feature point and each pixel within the search area.
According to certain example embodiments, the movement detecting apparatus further comprises imaging means for inputting images in time sequence.
According to certain example embodiments, the search area deciding means decides the search area on the basis of an amount of movement obtained from an image of a previous frame and the image of the current frame.
According to certain example embodiments, the feature point extracting means extracts a feature point of a spot having a large change in brightness from a plurality of areas in the image of the current frame in which coordinates in one direction coincides with each other, the search area deciding means decides the search area for the correlation operation per area of an image of a subsequent frame and the correlation operation means implements the correlation operation per area.